Click HERE to return to the contents page
Click HERE to return to the contents page
Natural Language Processing Guide
The Natural Language Processing (NLP) guide is written to help you understand how you can integrate NLP and customize Kore bots to interact using common everyday language from customers to perform web services between one or more applications. The chatbot we use at Kore, is called Kora. Kora interacts with customers simulating human requests and responses to collect data to execute a bot message alert or action for a web service. But before we get started, you need to understand the Kore terminology for bots and NLP.
Kore Bot Terminology
Bot – A bot is a simplified web application that can perform some actions and provide alerts messages from web services. The Weather Underground site, for example, can give you the current weather, the 3-day forecast and the ten-day forecast. This is weather information that users often need and would be configured for a bot. Conversely, Weather Underground can also provide historical temperature values, which is not a bot feature.
A bot has a collection of actions and alerts tied to a specific website, such as Weather Underground or Twitter, or a collection of tasks across different websites. The sites do not even have to be related.
Bots are defined using the Bot Builder Tool. To define a bot, you must decided which actions your bot will support. If you define too many actions, you can drown the user in the actions that they can do. And of course, your bot is limited by the actions that the web service API will support.
Action – This is what a bot can do for you now, like create a lead in Salesforce, or to get you a 3-day weather forecast for your location. Actions have two major components:
• Goal – This is the goal or function to accomplish using the bot, sometimes called intent in other platforms. For example, a goal can be to get a 3-day weather forecast for your location. The goal for an action should also be the name of the action.
• Fields – These are the arguments for the goal, for example, the location that you want the weather forecast for. Kore uses the term fields instead of arguments because basically what the customer is doing is filling in the fields of a web form using NLP instead of a web interface.
Getting the bot to tell you the 3-day forecast is a goal. Getting that forecast for Seattle is a specific location field needed to carry out this task.
Alert – An alert is a recurring notification of some change in information, such as new stories on a website such as CNN about solar power. Alerts can also let you know when someone tweets from the people you follow. An alert is setup in the present to act in the future to notify you of a change. It does not do anything about that change. Like actions, alerts also have goals and fields.
Flow – At Kore, a flow is a connection, or mapping, between an alert and one or more actions. Data from the alert can be pre-populated into an action. In a flow, a user can flow from an alert to a choice of one or more actions, and may need to provide additional data to execute the action. For example, in a flow, an alert message for a new tweet from Twitter can offer the user a choice of actions about that tweet, such as create a JIRA issue with alert data pre-populated in the action for JIRA Assigned to, Description, and Date/Time fields.
NLP – Natural Language Processing is a field of computer science, artificial intelligence, and computational linguistics designed to interpret and interact using common, everyday language between computers and humans. Kore uses NLP to interact with bot users, for example, to set up an alert message, or perform an action based on an alert message using messaging and chat channels.
Kora – The generic natural language interpreter for the Kore Bots Platform is called Kora. Kora is like a queen bot that rules over all other bots. A user is basically always talking to Kora, and based on the conversation, Kora determines which bot to use and what it should do. Even when the user clicks on a specific bot icon, the user is actually talking to Kora.
Synonyms – When defining a bot using NLP, you will create a list of synonyms to help Kora improve recognition of a bot name, alert name, or action name. For example, you might name an action, “Create Lead.” Kora looks for that name in the user’s input. Because users might provide a name that means the same thing, like “Make a New Lead,” or “Build Lead,” you would define a list of synonyms for words associated with your bot to improve recognition of user input. Some synonyms are predefined in Bot Builder Tool, and you can edit, add, or delete synonyms.
Pattern – Even if you define many synonyms, sometimes there are idiomatic ways of saying something that Kora would not know. For example, if your task name is “Get Current Weather,” a user might input “How’s it Outside today?” This idiom is a unique pattern of words that can be mapped to a specific action. You can define patterns for your bot task so that Kora is aware of common idioms as well as synonyms for improved recognition.
When applications were all done with GUI’s, words were not that important. You could write lots of words on a menu label and picking which words to use was not critical. Maybe a menu choice was “top opportunities for won/lost cases over $n”.
With the advent of natural language, words become critically important. You need to change your mindset for several reasons.
1. Users have to type in words and will want to minimize what they have to type. Also users have to remember roughly the names of your actions and shorter names are easier to remember. Pity the poor user who has to remember and type “Microsoft Dynamics CRM Tasks” whenever they want to reference that bot. “Microsoft CRM” should have been sufficient.
2. Users will be thinking from their perspective. Naming an action “get your tasks” is from the wrong perspective and should be “get my tasks”.
3. Speech to text will have issues with some kinds of things, like “SRP – C4C”. E.g., the user will be saying hyphen, but the system won’t know whether spaces exist or not. Even without speech to text, users will probably have difficulty remembering that there are symbols like _ or – and where to apply them in their typing. Avoid punctuation, hyphens, and underscores in words. Also avoid cutesy symbols like * and & and $. They have the same problems of pronunciation, recognition, and remembrance.
4. Avoid a slash symbol ” / ” in any name. For example, Opportunity Closed Won/Lost is not something a user will say. They might say “won or lost opportunity closed,” but really it is probably enough to say “opportunity closed” as the name for an alert, and in the help text, you can explain it is won or lost.
Similarly, don’t use parentheses ” ( ) ” in a name since the user won’t. For example, Create New HTTP(Custom) Check is a poor choice for action name.
Digits may be tolerable, but are another potential source of confusion. Ideally you would just use letters in your words.
Avoid apostrophes in any name. Contractions are always expanded out by Kora, so you never see “what’s” directly, it becomes “what is”. And possessives are also split out. Possessives don’t pronounce well (dogs vs dog’s vs dogs’) so speech to text will have potential issues.
Kora has to understand what the user says, and if you reuse the name of the bot in an action or the part of the name of an action in a field, it can be confusing and redundant. If the user is currently using a salesforce bot, naming an action “create salesforce lead” is unnecessary. Just “create lead” is the correct name. Then for a field argument of the person’s name, don’t call it “lead name.” Just use “name” because it’s obvious that it’s a lead name.
When you plan on using NLP for your bots, alert tasks, and action tasks, it is important that you follow some best practices names. Since Kora must be able to interpret the user input and locate the bot or task that the user wants, you should follow some best practices when it comes to the name of your bot or task.
The name of your bot name should be:
• Relatively short – Only one or two words. Just enough to identify the bot uniquely, but not require a lengthy user input.
• No punctuation – Do not use punctuation, hyphens, underscores, or any special keyboard characters. The only real exception to the “no punctuation” rule is a name like Flowers.com since everyone recognizes and remembers periods in a web-name, pronounced as “flowers dot com.”
When you define an action, you should name the action by the goal it is designed to accomplish, for example, “Get 3 day forecast.” You might think it sound nice to name it “Get the 3 day forecast,” or “Get me the 3 day weather forecast,” but adding “the” and “me” does not really contribute anything to the meaning for Kora since the user is already talking to Kora. Typically, you should define the minimal name needed to convey the goal.
But what about “Get forecast” since that is even shorter? Yes, it is shorter but then the range of the forecast would have to be specified by the user. By specifying the range in the name tells the user what range is available and is how the website itself organizes its data. Similarly, you should name a task “Create Lead” vs “Create a Lead.”
For actions, when you define a bot in Bot Builder Tool, the name of the action is also the “command” for that action. Bot commands should start with an infinitive verb. For example, when defining an action name, expect the user to fill in the sentence, “The goal is to xxx” where xxx is your action name. Typically, after the verb, you would provide the object of the verb, possibly with an adjective description, for example, “Show opportunities” or “Show top opportunities.”
You must define an object for most verbs in an action name to distinguish exactly what action is to take place. An action called “Update” doesn’t tell you enough. Many tasks can be named “Update.” You should define what is to be updated, for example, “Update Ticket.” However, some actions may not need an object, for example, Tweet, Retweet, and Follow have unique verbs that don’t require objects in the action name for a Twitter bot. We don’t use “Follow User” as this is somewhat redundant since that’s what Follow does.
Kora needs to be able to match what the user enters with your task name. Users are not likely to say “Follow user @bab,” they will just say “follow @bab.” For action names, do not require users to input more words than they need to, so keep the action name brief. Kora interprets the user input and then matches that input to the list of action names defined. If you have an action “Create lead,” Kora can parse the user input from “I want a lead for Jon Brown created,” and determine that the user wants to create a lead.
If your action name was “Create a Lead” vs “Create Lead,” Kora could still interpret the action name correctly as Kora assigns little or no value to noise words such as determiners, prepositions, and pronouns in your action name and focuses primarily on the significant words.
For actions, you are writing a command sentence. Except for self-explaining verbs like “Tweet,” you need specify an object for your verb. For example, “Permanently delete” is a poor action name for an action in Google Tasks because:
• it does not state what is being deleted
• it does not start with a command verb
• Users are not going to use “Permanently”
Since alerts only act in the future they are not labeled like actions. Imagine you are filling in the sentence “Notify me about xxx” or “Tell me when xxx”. The alert name is the object of the preposition “about” or the subject and verb of the clause after when. So typically an alert name is a noun or a noun with modifiers. “Lead updates” or “Status changes” are fine alert names. “Changes in Status” is possible but becomes wordier than users want to type and it’s pithier and simpler to write “Status changes” (which can work with either the about preposition or the when clause).
Don’t name an alert using the word alert. Don’t create an alert in Loggly named “Loggly Alert”. It is hard to distinguish and its not very informative.
Don’t name an alert like an action by starting with a verb. “Get news updates” – should just be “news updates”.
Most actions require fields, or arguments. These arguments typically round out the command sentence with the “who/what/where/when/why/how/how-much” kinds of data.
For our 3-day forecast task, an obvious field that we need is “Location.” Not “city where you want the weather from.” And not “Weather location” because “weather” is redundant in this action. It’s already all about the weather.
Field names are typically the objects of a some phrase like “for Jon Brown” “of Kore, Inc.” We require that fields have names and we require your field names to be nouns or modified nouns. A modified noun is something like an adjective before a noun such as “major opportunities” or a noun with a past-participle verb after it, for example, “job created.”
Ideally, the name is 1 or 2 words only. Again, this is because the user may need to refer to these names and won’t want to type much or remember lengthy names.
When Kora confirms with the user what data has been recognized, she will say something like “I understand for location you have given New York City, for execution date you have said May 3, 2016, …” The field names make it obvious what data Kora is referring to and users become familiar with the names. An experienced user can then use the names to fully specify an input request like “tell me the weather where the location is Seattle with execution date of three days from now.”
Actually, naming the fields in the input data is only one way a user can tell Kora what values the task fields have. Another way is by telling the bot tool exactly what kind of data a field is. For example, the location field might be a datatype of <location> and the execution date field probably has a datatype of <date> which narrows the difficulty of the interpretation for Kora. The user could then say “tell me weather for New York assuming 3 days from now” and Kora could sort it out.
Just as actions may have idioms to represent the action, fields may involve idioms to find the data or remap the data. For an action of purchasing bottles of wine, a wine shop website may want to know how many bottles of wine you want. But the user may say they want 5 cases of something. The simple field amount would probably be of type <number>, and works fine when the user says they want 5 shiraz. But cases is a numeric idiom meaning 12 bottles in a case, and so the developer may write a special field pattern that detects a number followed by the word cases and will convert that into the correct value to save with the field.
The bot tool has a predefined set of datatypes you can use that clue Kora into knowing how to find field data and what legal values are. But developers can extend that as well. A pizza bot probably wants to define the range of legal pizza toppings and Kora doesn’t know anything about that. You can define static or dynamic dropdown menus in the bot tool. A static dropdown is merely a list of legal choices like pepperoni or garlic. When Kora is examining the user’s input, she can consult the list and see what has been mentioned in input.
Dynamic drop-downs are similar to static ones, except that the values come from querying a website dynamically and the query may be based on the values of prior fields that have been processed.
Interactive Kora
Whenever Kora does NOT understand a field value from the given input (or the value was not supplied), Kora will prompt the user for the data explicitly if the data essential (you mark data as required or not in the bot tool). And Kora can confirm the values she did detect and allow the user to amend them if she got them wrong. So one way or another, eventually she can get all the data needed to carry out an action or alert.
